Apparatus and method of recognizing movement of subject

ABSTRACT

Provided is an apparatus and method of recognizing a movement of a subject. The apparatus includes a light source configured to emit light to the subject and an image sensor configured to receive light reflected from the subject. The apparatus includes a processor configured to detect a pixel that is receiving the reflected light, the pixel being included in a pixel array of the image sensor. The processor is configured to track the movement of the subject based on a change in a position of the detected pixel.

RELATED APPLICATION

This application is a divisional application of and hereby claimspriority under 35 U.S.C. §§ 120,121 to U.S. application Ser. No.14/624,715 filed Feb. 18, 2015, which claims the benefit of KoreanPatent Application No. 10-2014-0110958, filed on Aug. 25, 2014, in theKorean Intellectual Property Office, the entire contents of each ofwhich are hereby incorporated herein by reference.

BACKGROUND 1. Field

At least one example embodiment relates to apparatuses and/or methods ofrecognizing a movement of a subject, and more particularly, toapparatuses and/or methods of recognizing a three-dimensional (3D)movement of a subject by using a light source and an image sensor.

2. Description of the Related Art

According to recent developments in image processing technology, animage processing apparatus recognizes a movement of a subject by usingthree-dimensional (3D) information, for example, by using a depth cameraor a stereo camera. In particular, the depth camera uses a time offlight (ToF) scheme that measures a time taken for emitted light to bereflected from an object.

As such, in order to perform image processing using 3D information,power used for various modulation processes is consumed or an additionalsoftware (SW) resource such as a filter used for various imageprocessing tasks is required. That is, in order to more preciselyrecognize a movement of a subject, many processing tasks using hardwareor software are required, and thus increased processing power isconsumed.

SUMMARY

At least one example embodiment provides apparatuses and/or methods ofrecognizing a movement of a subject by using a light source and an imagesensor.

At least one example embodiment provides computer-readable recordingmedia having embodied thereon programs for executing the methods incomputers. Additional aspects will be set forth in part in thedescription which follows and, in part, will be apparent from thedescription, or may be learned by practice of example embodiments.

According to at least one example embodiment, an apparatus forrecognizing a movement of a subject includes: a light source configuredto emit light to the subject; an image sensor configured to receivelight reflected from the subject; and a processor configured to detect apixel that is receiving the reflected light, the pixel being included ina pixel array of the image sensor. The processor is configured to trackthe movement of the subject based on a change in a position of thedetected pixel.

According to at least one example embodiment, the apparatus includes afilter configured to filter the reflected light to transmit light havinga desired wavelength. The processor is configured to detect a pixelusing the transmitted light having the desired wavelength.

According to at least one example embodiment, the processor isconfigured to set a reference value for an intensity of light, anddetect a pixel having an intensity that is greater than the referencevalue.

According to at least one example embodiment, the pixel is in aplurality of pixels in the pixel array, and the processor is configuredto detect a pixel is located at a central position among the pluralityof pixels.

According to at least one example embodiment, the processor isconfigured to calculate a distance by which the subject moves based onan amount of the change in the position of the detected pixel.

According to at least one example embodiment, the apparatus includes alens configured to collect the reflected light such that the imagesensor receives the reflected light collected by the lens.

According to at least one example embodiment, the apparatus includes arecognizer configured to recognize a pattern according to the change inthe position of the detected pixel.

According to at least one example embodiment, the light source isconfigured to emit light that passes through an opening.

According to at least one example embodiment, the apparatus includes adisplay configured to display a pattern according to the change in theposition of the detected pixel.

According to at least one example embodiment, a method of recognizing amovement of a subject includes emitting, by a light source, light to thesubject and receiving, by an image sensor, light reflected from thesubject. The method includes detecting a pixel that is receiving thereflected light, the pixel being included in a pixel array of the imagesensor. The method includes tracking the movement of the subject basedon a change in a position of the detected pixel.

According to at least one example embodiment, the method includesfiltering the reflected light to transmit light having a desiredwavelength, and the detecting comprises detecting a pixel using thetransmitted light having the desired wavelength.

According to at least one example embodiment, the detecting comprisessetting a reference value for an intensity of light, and detecting apixel having an intensity that is greater than the reference value.

According to at least one example embodiment, the pixel is in aplurality of pixels in the pixel array, and the detecting comprisesdetecting a pixel that is located at a central position among theplurality of pixels.

According to at least one example embodiment, the method includescalculating a distance by which the subject moves based on an amount ofthe change in the position of the detected pixel.

According to at least one example embodiment, the receiving lightreflected from the subject comprises receiving the light reflected fromthe subject through a lens.

According to at least one example embodiment, the method includesrecognizing a pattern according to the change in the position of thedetected pixel.

According to at least one example embodiment, the light emitted to thesubject is light that passes through an opening.

According to at least one example embodiment, the method includesdisplaying a pattern according to the change in the position of thedetected pixel.

According to at least one example embodiment is directed to anon-transitory computer-readable recording medium having embodiedthereon a program for executing the above method.

BRIEF DESCRIPTION OF THE DRAWINGS

These and/or other aspects will become apparent and more readilyappreciated from the following description of the example embodiments,taken in conjunction with the accompanying drawings in which:

FIG. 1 is a block diagram illustrating an apparatus for recognizing amovement of a subject, according to at least one example embodiment;

FIG. 2 is a view for explaining an example where the apparatus ismounted in a wearable device and tracks a movement of a subject,according to at least one example embodiment;

FIGS. 3A and 3B are views for explaining a method performed by theapparatus to track a movement of a subject along a z-axis of FIG. 2,according to at least one example embodiment;

FIGS. 4A and 4B are views for explaining a method performed by theapparatus to track a movement of a subject along an x-axis of FIG. 2,according to at least one example embodiment;

FIG. 5 is a view for explaining a process of recognizing a change in aposition of a subject by combining the methods of FIGS. 3 and 4,according to at least one example embodiment;

FIGS. 6A and 6B are views for explaining a method performed by theapparatus to track a movement of a subject along a y-axis of FIG. 2,according to at least one example embodiment;

FIG. 7 is a diagram for explaining a method performed by the apparatusto detect a specific pixel that is located in an image sensor, accordingto at least one example embodiment;

FIG. 8 is a block diagram illustrating the apparatus, according to atleast one example embodiment;

FIG. 9 is a view for explaining the apparatus including a filter,according to at least one example embodiment;

FIGS. 10A and 10B are views for explaining a function of a recognizerthat is included in the apparatus, according to at least one exampleembodiment;

FIG. 11 is a flowchart of a method of recognizing a movement of asubject, according to at least one example embodiment; and

FIG. 12 is a flowchart of an operation of detecting a specific pixel ina pixel array that is included in the image sensor in the method of FIG.11, according to at least one example embodiment.

DETAILED DESCRIPTION OF EXAMPLE EMBODIMENTS

Inventive concepts will now be described more fully with reference tothe accompanying drawings, in which example embodiments of are shown.These example embodiments are provided so that this disclosure will bethorough and complete, and will fully convey inventive concepts of tothose skilled in the art. Inventive concepts may be embodied in manydifferent forms with a variety of modifications, and a few embodimentswill be illustrated in drawings and explained in detail. However, thisshould not be construed as being limited to example embodiments setforth herein, and rather, it should be understood that changes may bemade in these example embodiments without departing from the principlesand spirit of inventive concepts, the scope of which are defined in theclaims and their equivalents. Like numbers refer to like elementsthroughout. In the drawings, the thicknesses of layers and regions areexaggerated for clarity.

It will be understood that, although the terms first, second, etc. maybe used herein to describe various elements, these elements should notbe limited by these terms. These terms are only used to distinguish oneelement from another. For example, a first element could be termed asecond element, and, similarly, a second element could be termed a firstelement, without departing from the scope of example embodiments. Asused herein, the term “and/or” includes any and all combinations of oneor more of the associated listed items.

It will be understood that when an element is referred to as being“connected” or “coupled” to another element, it can be directlyconnected or coupled to the other element or intervening elements may bepresent. In contrast, when an element is referred to as being “directlyconnected” or “directly coupled” to another element, there are nointervening elements present. Other words used to describe therelationship between elements should be interpreted in a like fashion(e.g., “between” versus “directly between,” “adjacent” versus “directlyadjacent,” etc.).

Unless specifically stated otherwise, or as is apparent from thediscussion, terms such as “processing” or “computing” or “calculating”or “determining” or “displaying” or the like, refer to the action andprocesses of a computer system, or similar electronic computing device,that manipulates and transforms data represented as physical, electronicquantities within the computer system's registers and memories intoother data similarly represented as physical quantities within thecomputer system memories or registers or other such information storage,transmission or display devices.

Specific details are provided in the following description to provide athorough understanding of example embodiments. However, it will beunderstood by one of ordinary skill in the art that example embodimentsmay be practiced without these specific details. For example, systemsmay be shown in block diagrams so as not to obscure example embodimentsin unnecessary detail. In other instances, well-known processes,structures and techniques may be shown without unnecessary detail inorder to avoid obscuring example embodiments.

In the following description, illustrative embodiments will be describedwith reference to acts and symbolic representations of operations (e.g.,in the form of flow charts, flow diagrams, data flow diagrams, structurediagrams, block diagrams, etc.) that may be implemented as programmodules or functional processes include routines, programs, objects,components, data structures, etc., that perform particular tasks orimplement particular abstract data types and may be implemented usingexisting hardware in existing electronic systems (e.g., electronicimaging systems, image processing systems, digital point-and-shootcameras, personal digital assistants (PDAs), smartphones, tabletpersonal computers (PCs), laptop computers, etc.). Such existinghardware may include one or more Central Processing Units (CPUs),digital signal processors (DSPs),application-specific-integrated-circuits (ASICs), field programmablegate arrays (FPGAs) computers or the like.

Although a flow chart may describe the operations as a sequentialprocess, many of the operations may be performed in parallel,concurrently or simultaneously. In addition, the order of the operationsmay be re-arranged. A process may be terminated when its operations arecompleted, but may also have additional steps not included in thefigure. A process may correspond to a method, function, procedure,subroutine, subprogram, etc. When a process corresponds to a function,its termination may correspond to a return of the function to thecalling function or the main function.

As disclosed herein, the term “storage medium”, “computer readablestorage medium” or “non-transitory computer readable storage medium” mayrepresent one or more devices for storing data, including read onlymemory (ROM), random access memory (RAM), magnetic RAM, core memory,magnetic disk storage mediums, optical storage mediums, flash memorydevices and/or other tangible or non-transitory machine readable mediumsfor storing information. The term “computer-readable medium” mayinclude, but is not limited to, portable or fixed storage devices,optical storage devices, and various other tangible or non-transitorymediums capable of storing, containing or carrying instruction(s) and/ordata.

Furthermore, example embodiments may be implemented by hardware,software, firmware, middleware, microcode, hardware descriptionlanguages, or any combination thereof. When implemented in software,firmware, middleware or microcode, the program code or code segments toperform the necessary tasks may be stored in a machine or computerreadable medium such as a computer readable storage medium. Whenimplemented in software, a processor or processors may be programmed toperform the necessary tasks, thereby being transformed into specialpurpose processor(s) or computer(s).

The terminology used herein is for the purpose of describing particularembodiments only and is not intended to be limiting. As used herein, thesingular forms “a”, “an” and “the” are intended to include the pluralforms as well, unless the context clearly indicates otherwise. It willbe further understood that the terms “includes”, “including”,“comprises” and/or “comprising,” when used in this specification,specify the presence of stated features, integers, steps, operations,elements, and/or components, but do not preclude the presence oraddition of one or more other features, integers, steps, operations,elements, components, and/or groups thereof.

Unless otherwise defined, all terms (including technical and scientificterms) used herein have the same meaning as commonly understood by oneof ordinary skill in the art to which inventive concepts belong. It willbe further understood that terms, such as those defined in commonly useddictionaries, should be interpreted as having a meaning that isconsistent with their meaning in the context of the relevant art andwill not be interpreted in an idealized or overly formal sense unlessexpressly so defined herein.

As used herein, the term “and/or” includes any and all combinations ofone or more of the associated listed items. Expressions such as “atleast one of,” when preceding a list of elements, modify the entire listof elements and do not modify the individual elements of the list.

Inventive concepts will now be described more fully with reference tothe accompanying drawings, in which example embodiments are shown.

FIG. 1 is a block diagram illustrating an apparatus 100 for recognizinga movement of a subject, according to at least one example embodiment.Hereinafter, the apparatus 100 for recognizing a movement of a subjectwill be referred to as the apparatus 100 for convenience. According toan embodiment, the apparatus 100 may include a light source 110, a lens120, an image sensor 130, and a processor 140. It should be understoodthat the apparatus 100 is not limited to the elements shown in FIG. 1.Accordingly, it will be understood by one of ordinary skill in the artthat the apparatus 100 may further include general-purpose elementsother than or in addition to the elements shown in FIG. 1.

According to at least one example embodiment, the light source 110 mayemit light to the subject. For example, the light source 110 may be alight-emitting diode (LED) that may emit light having a near-infrared(NIR) wavelength of about 850 nm, which is not detected by the human eyefor safety, or a laser diode (LD). However, example embodiments are notlimited to one wavelength band and one type of light source. Also, thelight source 110 may emit light at a preset angle, and the light may belight that passes through an opening (e.g., a slit). Also, the lightsource 110 may emit light having a desired (or alternatively,predetermined) wavelength such as infrared light.

According to at least one example embodiment, when the light emitted bythe light source 110 is reflected by the subject, the lens 120 maycollect part of the reflected light. According to at least one exampleembodiment, the lens 120 may be a convex lens and may send the collectedlight to the image sensor 130.

According to at least one example embodiment, the image sensor 130 mayreceive the light collected by the lens 120. The image sensor 130 mayinclude a pixel array, and a specific pixel in the pixel array mayreceive the light collected by the lens 120. According to at least oneexample embodiment, the image sensor 130 may be a complementarymetal-oxide-semiconductor (CMOS) image sensor (CIS) and may include atwo-dimensional position sensitive detector (2D-PSD) or a 2D-photodiode(PD).

According to at least one example embodiment, the processor 140 maydetect a pixel for receiving the light collected by the lens 120 in thepixel array of the image sensor 130 and may recognize a position of thesubject. That is, since a position of the pixel for receiving the lightreflected from the subject and collected by the lens 120 may varyaccording to the position of the subject, the processor 140 mayrecognize the position of the subject by detecting the pixel forreceiving the reflected and collected light in the pixel array that is a2D pixel array. According to at least one example embodiment, when theprocessor 140 detects a specific pixel for receiving light, theprocessor 140 detects the specific pixel for receiving light in theimage sensor 130 and recognizes a position of the specific pixel in thepixel array. Accordingly, the position of the detected pixel and theposition of the subject may correspond to each other in a one-to-onemanner.

Also, according to at least one example embodiment, the processor 140may track a movement of the subject based on a change in the position ofthe detected pixel. That is, the processor 140 may detect the change inthe position of the pixel for receiving the reflected and collectedlight according to the movement of the subject and may track themovement of the subject by using the change in the position of thedetected pixel.

According to at least one example embodiment, the processor 140 may seta critical value (or reference value) for an intensity of light and maydetect a pixel for receiving light having a greater intensity than theset critical value from among the collected light. Alternatively, when aplurality of pixels for receiving the collected light are provided inthe pixel array, the processor 140 may detect a pixel that is located ata central position from among the plurality of pixels and may recognizethe position of the subject.

Also, according to at least one example embodiment, the processor 140may set an optimal resolution which the image sensor 130 may recognizeby adjusting a distance between the light source 110 and the subject andan angle at which the light source 110 emits the light to the subject.That is, the processor 140 may adjust a ratio of a distance by which thesubject moves to the amount of the change in the position of the pixelby adjusting the angle at which the light is emitted.

Also, according to at least one example embodiment, the processor 140may calculate the distance by which the subject moves based on theamount of the change in the position of the detected pixel, which willbe explained below in detail with reference to FIGS. 3A and 3B.

FIG. 2 is a view illustrating an example where the apparatus 100 ismounted on a wearable device and tracks a movement of a subject,according to at least one example embodiment. Although the apparatus 100is mounted on a side surface of the wearable device 201 in FIG. 2,example embodiments are not limited thereto and the apparatus 100 may bemounted on a portable device such as a smart phone and on any surface ofthe portable device. Also, although the processor 140 of FIG. 1 is notshown in FIG. 2, the processor 140 is provided in the apparatus 100.

As shown in FIG. 2, the light source 110 may emit light that passesthrough a slit to a subject 210, and the emitted light may be reflectedfrom the subject 210. Next, the reflected light may be collected by thelens 120 (not shown) and may be focused on the image sensor 130.

Accordingly, the processor 140 may detect a pixel for receiving thecollected light in the pixel array that is included in the image sensor130, may detect a change in a position of the pixel for receiving thecollected light according to a movement of the subject 210, and maytrack the movement of the subject 210 by using the change in theposition of the detected pixel.

That is, according to at least one example embodiment, the subject 210may move along x, y, and z-axes, which are three-dimensional (3D)coordinate axes. For example, the subject 210 moves along the x andz-axes to form a sine wave as shown in FIG. 2. While the subject 210moves to form the sine wave, the processor 140 may detect the change inthe position of the pixel according to the movement of the subject 210and may display a pattern according to the change in the position of thedetected pixel on a display unit of a wearable device 201. Also, theapparatus 100 may display the pattern according to the change in theposition of the detected pixel by using a separate display unit.

Accordingly, the apparatus 100 may include a light source and an imagesensor and may track a movement of a subject by detecting a pixel of theimage sensor, position of the pixel corresponding in a one-to-one mannerto a position of the subject. Thus, the apparatus 100 may track themovement of a subject without performing image processing such as depthcalculation or image filtering. Consequently, power consumption may bereduced due to the exclusion of hardware or software used for suchoperations.

FIGS. 3A and 3B are views for explaining a method performed by theapparatus 100 to track a movement of a subject along the z-axis of FIG.2, according to at least one example embodiment. FIGS. 3A and 3Billustrate the apparatus 100 seen in a negative direction (opposite to adirection marked by an arrow) of the x-axis of FIG. 2. The followingwill be explained on the assumption that the subject is a finger. Also,although each element of the apparatus 100 is larger than the finger inFIGS. 3A and 3B for convenience, a ratio of a size of the finger to asize of the apparatus 100 is not limited to that in FIGS. 3A and 3B. Inother example embodiments, a ratio of a size of a subject to each ofelements of the apparatus 100 is not limited to that shown in FIGS. 3Aand 3B.

Referring to FIG. 3A, the light source 110 emits light that passesthrough a slit to the finger at a desired (or alternatively,predetermined) angle. Part of the emitted light is scattered andreflected from the finger. The scattered and reflected light iscollected by the lens 120 and is focused on a position 320 of a pixel ofthe image sensor 130 including the pixel array.

Accordingly, the processor 140 may detect the pixel for receiving thecollected light that is located at the position 320 in the image sensor130 and may recognize a position 310 of the finger by detecting thepixel that is located at the position 320 corresponding to the position310 of the finger. A process performed by the processor 140 to detectthe pixel that is located at the position 320 in the image sensor 130will be explained below in detail with reference to FIG. 7.

Referring to FIG. 3B, the finger moves from the position 310 of FIG. 3Ato a position 330. Like in FIG. 3A, part of light emitted from the lightsource 110 is scattered and reflected from the finger. The scattered andreflected light is collected by the lens 120 and is focused on a pixelthat is located at a position 340 in the image sensor 130.

Accordingly, the processor 140 may detect the pixel for receiving thecollected light that is located at the position 340 in the image sensor130 and may recognize the position 330 of the finger by detecting thepixel that is located at the position 340 corresponding to the position330 of the finger. Also, as the finger moves from the position 310 tothe position 330, the processor 140 may detect that the pixel forreceiving the collected light in the image sensor 130 changes from theposition 320 to the position 340, may detect a change in a position ofthe pixel for receiving the collected light according to a movement ofthe finger, and may track the movement of the finger by using thedetected change in the position of the pixel. Also, the processor 140may recognize that as the finger moves farther away from the lightsource 110, the position of the pixel for receiving the collected lightin the image sensor 130 moves closer to the light source 110.

Also, the processor 140 may calculate the amount of a change in aposition of the finger by using the amount of the change in the positionof the pixel. As shown in FIG. 3B, when the amount of the change in theposition of the finger is ΔZ, the amount of the change in the positionof the pixel is ΔY, an angle between the light which the light source110 emits and a horizontal surface is n, and a magnification of the lens120 is m, the amount of the change in the position of the finger ΔZ iscalculated as follows:

$\begin{matrix}{{\Delta\; Z} = {\frac{\cos\; n}{m \times \sin\; n} \times \Delta\;{y.}}} & (1)\end{matrix}$

However, Equation 1 contains desirable values, and thus an error due torefraction and scattering in air needs to be considered.

FIGS. 4A and 4B are views for explaining a method performed by theapparatus 100 to track a movement of a subject along the x-axis of FIG.2, according to at least one example embodiment. FIGS. 4A and 4Billustrate the apparatus 100 seen in a negative direction of the y-axisof FIG. 2. The following will be explained on the assumption that thesubject is a finger.

Referring to FIG. 4A, the light source 110 (not shown) emits light thatpasses through a slit to the finger at a desired (or alternatively,predetermined) angle. Part of the emitted light is scattered andreflected from the finger. The scattered and reflected light iscollected by the lens 120 and is focused on a pixel that is located at aposition 420 in the image sensor 130 including the pixel array that is a2D pixel array.

Accordingly, the processor 140 may detect the pixel for receiving thecollected light that is located at the position 420 in the image sensor130 and may recognize a position 410 of the finger by detecting thepixel that is located at the position 420 corresponding to the position410 of the finger. A process performed by the processor 140 to detectthe pixel that is located at the position 420 in the image sensor 130will be explained below in detail with reference to FIG. 7.

Referring to FIG. 4B, the finger moves from the position 410 of FIG. 4Ato a position 430. Like in FIG. 4A, part of light emitted from the lightsource 110 (not shown) is scattered and reflected from the finger. Partof the scattered and reflected light is collected by the lens 120 and isfocused on a pixel that is located at a position 440 in the image sensor130.

Accordingly, the processor 140 may detect the pixel for receiving thecollected light that is located at the position 440 in the image sensor130 and may recognize the position 440 of the finger by detecting thepixel that is located at the position 440 corresponding to the position430 of the finger. Also, as the finger moves from the position 410 tothe position 430, the processor 140 may detect that the pixel forreceiving the collected light in the image sensor 130 changes from theposition 420 to the position 440, may detect a change in a position ofthe pixel for receiving the collected light in the image sensor 130, andmay track a movement of the finger by using the detected change in theposition of the pixel. Also, the processor 140 may recognize that as thefinger moves in a positive direction of the x-axis, the position of thepixel for receiving the collected light in the image sensor 130 changesin a negative direction of the x-axis.

FIG. 5 is a view for explaining a process of recognizing a change in aposition of a subject by combining the methods of FIGS. 3 and 4,according to at least one example embodiment. The following will beexplained on the assumption that the subject is a finger. Also, theimage sensor 130 and the light source 110 of the apparatus 100 are shownand other elements are not shown for convenience. Also, scattering andreflecting from the finger and light collection using the lens 120 (notshown) will not be described here.

As shown in FIG. 5, light is emitted from the light source 110, thefinger moves from a position 510 to a position 520 and then from theposition 520 to a position 530 in order to form a

. As the finger moves, a position of a pixel for receiving light in theimage sensor 130 including the pixel array also changes. Accordingly,when the finger moves from the position 510 to the position 520, thepixel for receiving light changes from a position 540 to a position 550.Also, when the finger moves from the position 520 to the position 530,the pixel for receiving light changes from the position 550 to aposition 560.

Accordingly, the apparatus 100 may recognize that the finger moves fromthe position 510 through the position 520 to the position 530 bydetecting that the pixel for receiving light changes from the position540 through the position 550 to the position 560.

FIGS. 6A and 6B are views for explaining a method of recognizing amovement of a subject along the y-axis of FIG. 2, according to at leastone example embodiment. FIGS. 6A and 6B illustrate the apparatus 100seen in a negative direction of the x-axis of FIG. 2. The following willbe explained on the assumption that the subject is a finger.

In FIG. 6A, the light source 110 emits light that passes through a slitto the finger at a desired (or alternatively, predetermined) angle.However, since the finger is located over the emitted light, the lightis not scattered and reflected. Accordingly, the image sensor 130 doesnot receive light that is reflected or collected by the lens 120.

In FIG. 6B, the finger moves in a negative direction of the y-axis to aposition 620. As the finger moves, the light emitted from the lightsource 110 is scattered and reflected from the finger, and the reflectedlight is collected by the lens 120 and is focused on the image sensor130.

Accordingly, the processor 140 may detect a movement of the finger alongthe y-axis according to whether a pixel for receiving the collectedlight exists in the image sensor 130. That is, when a two-step sequenceof no pixel detection and pixel detection occurs during a desired (oralternatively, predetermined) period of time, the processor 140recognizes that the finger moves in the negative direction of they-axis. Also, when a two-step sequence of pixel detection and no pixeldetection occurs for a desired (or alternatively, predetermined) periodof time, the processor 140 recognizes that the finger moves in apositive direction of the y-axis.

FIG. 7 is a diagram for explaining a method performed by the apparatus100 to detect a specific pixel for receiving light that is located inthe image sensor 130, according to at least one example embodiment. Thatis, in detail, the processor 140 that is provided in the apparatus 100detects the specific pixel for receiving light that is located in theimage sensor 130.

According to at least one example embodiment, the image sensor 130 mayinclude the pixel array that is a 2D pixel array as shown in FIG. 7. Thenumber or shapes of pixels are not limited to those of FIG. 7. The imagesensor 130 may receive light collected by the lens 120 of FIG. 1. Asshown in FIG. 7, pixels of a desired (or alternatively, predetermined)area 710 may receive light. Although the desired (or alternatively,predetermined) area 710 for receiving light has an oval shape in FIG. 7,example embodiments are not limited thereto.

According to at least one example embodiment, the pixels that areincluded in the desired (or alternatively, predetermined) area 710 forreceiving light receive pieces of light having different intensities. Inan upper graph 720, the x-axis represents positions of pixels in ahorizontal direction and the y-axis represents the intensities of thepieces of light received by the pixels. Also, in a right graph 730, thex-axis represents positions of pixels in a vertical direction and they-axis represents intensities of pieces of light received by the pixels.It is found from the upper and right graphs 720 and 730, each having aGaussian distribution, that an intensity of light in a middle portion isthe highest and intensities of light decrease away from the middleportion. The Gaussian distribution is also not limited to that of FIG.7.

According to at least one example embodiment, the processor 140 may seta critical value (or reference value) for an intensity of light and maydetect only pixels for receiving pieces of light having greaterintensities than the set critical value. The critical value (orreference value) may be user selected and/or based on empiricalevidence. Accordingly, the processor 140 may set a critical value 740 inthe upper graph 720 and may detect pixels for receiving pieces of lighthaving greater intensities than the set critical value 740 in a desired(or alternatively) predetermined portion 750. Also, the processor 140may set a critical value 760 in the right graph 730 and may detectpixels for receiving pieces of light having greater intensities than theset critical value 760 in a desired (or alternatively, predetermined)area 770. Accordingly, the processor 140 may detect pixels in an area780 where the desired (or alternatively, predetermined) portions 750 and770 of the upper and right graphs 720 and 730 are shared. That is, theprocessor 140 may set a critical value for an intensity of light, maydetect a pixel for receiving light having a greater intensity than theset critical value, and may recognize a position of a subjectcorresponding to a position of the detected pixel.

Also, according to at least one example embodiment, the processor 140may detect a pixel that is located at a central position from among aplurality of pixels for receiving light without using theafore-described method and may recognize a position of the pixel.

FIG. 8 is a block diagram illustrating the apparatus 100 according to atleast one example embodiment. The apparatus 100 may further include afilter 810, a recognizer 820, and a display unit 830 in addition to thelight source 110, the lens 120, the image sensor 130, and the processor140 of FIG. 1.

According to at least one example embodiment, the filter 810 may be afilter that may transmit therethrough only light having a desired (oralternatively, predetermined) wavelength. For example, when the lightsource 110 emits infrared light to a subject, the filter 810 mayselectively transmit therethrough infrared light from among lightreflected from the subject or light collected by the lens 120, whichwill be explained below in detail with reference to FIG. 9.

According to at least one example embodiment, the recognizer 820 mayrecognize a pattern according to a change in a position of a pixeldetected by the processor 140. That is, as the subject moves, theposition of the pixel for receiving light reflected from the subject inthe pixel array of the image sensor 130 may change, and the recognizer820 may recognize a pattern according to a change in the position of thepixel for receiving the reflected light, which will be explained belowin detail with reference to FIG. 10. The functions of recognizer 820 maybe performed by processor 140 or by a separate processor (not shown).

According to at least one example embodiment, the display unit 830 maydisplay the pattern according to the change in the position of the pixeldetected in the image sensor 130 by the processor 140. Also, the displayunit 830 may include a display panel (not shown). Various examples ofthe display panel may include a liquid crystal display (LCD) panel, anorganic light-emitting diode (OLED) display panel, an active-matrix(AM)-OLED panel, and a plasma display panel (PDP).

Also, according to at least one example embodiment, the apparatus 100may include a marker (not shown) that helps to more accurately recognizea movement of the subject. For example, the marker that is attached tothe subject may reduce scattering of light emitted from the light source110, and thus light reflected from the subject may be more accuratelyfocused on a specific pixel of the image sensor 130.

FIG. 9 is a view for explaining the apparatus 100 including a filter910, according to at least one example embodiment.

According to at least one example embodiment, the light source 110 mayemit infrared light to a subject. The emitted infrared light may bereflected from the subject, and the reflected light may be collected bythe lens 120 and may be focused on the image sensor 130. In this case,the filter 910 that is disposed between the lens 120 and the imagesensor 130 may selectively transmit therethrough only infrared lightfrom among the collected light, and the image sensor 130 may receiveonly the collected infrared light. In this case, the image sensor 130may be a sensor that selectively reacts to infrared light.Alternatively, unlike in FIG. 9, the filter 910 may be disposed betweenthe lens 120 and a subject 920 and the lens 120 may function as thefilter 910. According to at least one example embodiment, light otherthan infrared light may be used.

FIGS. 10A and 10B are views for explaining a function of the recognizer820 that is included in the apparatus 100, according to at least oneexample embodiment. Also, in FIG. 10A, only the image sensor 130 and thelight source 110 of the apparatus 100 are shown and other elements arenot shown for convenience.

FIG. 10A is a view illustrating pixels detected by the image sensor 130when a finger moves on a side surface of the apparatus 100 to form aletter ‘

’. Although the finger smoothly moves to form the letter ‘

’, actually, the pixels detected by the image sensor 130 may not besmooth. In this case, the recognizer 820 may recognize the letter (orsymbol) ‘

’ by recognizing a pattern according to a change in positions of thepixels detected by the image sensor 130. The recognizer 820 mayrecognize the letter ‘

’ by learning a pattern according to a change in positions of manypixels corresponding to the letter ‘

’. Although the letter ‘

’ is used in FIG. 10A, example embodiments are not limited thereto andany letter may be used.

Also, a user may set that the recognizer 820 may recognize a specificpattern. For example, the user may set that a movement of a subjectalong a y-axis is recognized as a ‘click’ input of the apparatus 100.That is, when the processor 140 detects that for a desired (oralternatively, predetermined) period of time, there is a pixel that isnot detected by the image sensor 130, is detected again, and is notdetected again, the recognizer 820 may recognize such a movement of thesubject along the y-axis as a ‘click’ input.

FIG. 10B is a view illustrating that the letter ‘

’ recognized by the recognizer 820 in FIG. 10A is displayed on a displayunit 1030 that is included in the apparatus 100, according to at leastone example embodiment.

The apparatus 100 according to at least one example embodiment may bemounted on a portable device such as a wearable device or a smart phone,and thus may be used as a motion recognition apparatus in any of varioususer interfaces such as a handwritten letter recognition input device ina near area. Also, the apparatus 100 may be used as a device fordetecting an absolute position of a specific object in a coordinatesystem in the industry.

FIG. 11 is a flowchart of a method performed by the apparatus 100 ofFIG. 1 to recognize a movement of a subject, according to at least oneexample embodiment.

In operation S1110, the light source 110 that is included in theapparatus 100 may emit light to the subject. The light source 110 mayemit light at a desired (or alternatively, preset) angle, and the lightmay be light that passes through a slit. Also, the light source 110 mayemit light having a desired (or alternatively, predetermined) wavelengthsuch as infrared light.

In operation S1120, the image sensor 130 that is included in theapparatus 100 may receive light reflected from the subject. Also, thelens 120 that is included in the apparatus 100 may collect part of thelight reflected from the subject, and the image sensor 130 may receivethe collected light. According to at least one example embodiment, theimage sensor 130 may include a pixel array, and a specific pixel in thepixel array may receive the light collected by the lens 120.

In operation S1130, the apparatus 100 may detect a pixel for receivingthe light that is reflected from the subject or collected by the lens120 in the pixel array of the image sensor 130. According to at leastone example embodiment, when the apparatus 100 detects a specific pixelfor receiving light, it may mean that the processor 140 may detect thespecific pixel for receiving light in the image sensor 130 and mayrecognize a position of the specific pixel in the pixel array.Accordingly, a position of the detected pixel and a position of thesubject may correspond to each other in a one-to-one manner. A processperformed by the apparatus 100 to detect the pixel for receiving lightwill be explained below in detail with reference to FIG. 12.

In operation S1140, the apparatus 100 may track a movement of thesubject based on a change in the position of the pixel detected inoperation S1130. That is, the apparatus 100 may detect the change in theposition of the pixel for receiving the reflected or collected lightaccording to the movement of the subject and may track the movement ofthe subject by using the change in the position of the detected pixel.Also, according to at least one example embodiment, the apparatus 100may calculate a distance by which the subject moves based on the amountof the change in the position of the detected pixel.

FIG. 12 is a flowchart of operation S1130 in the method of FIG. 11,according to at least one example embodiment.

In operation S1210, the apparatus 100 determines whether a specificpixel receives light that is reflected from the subject or light that iscollected by the lens 120 in the pixel array that is included in theimage sensor 130. If it is determined in operation S1210 that thespecific pixel does not receive the light, the method proceeds tooperation S1240. In operation S1240, it is determined that the specificpixel is not detected by the apparatus 100.

Otherwise, if it is determined in operation S1210 that the specificpixel receives the light, the method proceeds to operation S1220. Inoperation S1220, the apparatus 100 determines whether an intensity ofthe light received by the specific pixel is greater than a desired (oralternatively, predetermined) critical value (or reference value). Thedesired (or alternatively, predetermined) critical value may be presetby the apparatus 100. If it is determined in operation S1220 that theintensity of the light is not greater than the desired (oralternatively, predetermined) critical value, the method proceeds tooperation S1240. In operation S1240, it is determined that the specificpixel is not detected by the apparatus 100.

Otherwise, if it is determined in operation S1220 that the intensity ofthe light received by the specific pixel is greater than the desired (oralternatively, predetermined) critical value, the method proceeds tooperation S1230. In operation S1230, it is determined that the specificpixel is detected by the apparatus 100.

Also, according to at least one example embodiment, the apparatus maydetect a pixel that is located at a central position from among aplurality of pixels for receiving light without the afore-describedmethod and may recognize a position of the pixel.

As described above, according to at least one example embodiment, sincea position of a subject and a position of a pixel in an image sensorcorrespond to each other, a movement of the subject may be tracked bydetecting a change in the position of the pixel.

The device described herein may include a processor, a memory forstoring program data and executing it, a permanent storage unit such asa disk drive, a communications port for handling communication withexternal devices, and user interface devices, including a touch panel,keys, buttons, etc. When software modules or algorithms are involved,these software modules may be stored as program instructions orcomputer-readable codes executable on a processor on a computer-readablerecording medium. Examples of the computer-readable recording mediuminclude magnetic storage media (e.g., read-only memories (ROMs),random-access memories (RAMs) floppy disks, hard disks, etc.), andoptical recording media (e.g., compact disc (CD)-ROMs, or digitalversatile discs (DVDs)). The computer-readable recording medium can alsobe distributed over network-coupled computer systems so that thecomputer-readable code is stored and executed in a distributive manner.This media can be read by the computer, stored in the memory, andexecuted by the processor.

Inventive concepts may be described in terms of functional blockcomponents and various processing steps. Such functional blocks may berealized by any number of hardware and/or software components configuredto perform the specified functions. For example, inventive concepts mayemploy various integrated circuit (IC) components, e.g., memoryelements, processing elements, logic elements, look-up tables, and thelike, which may carry out a variety of functions under the control ofone or more microprocessors or other control devices. Similarly, wherethe elements of inventive concepts are implemented using softwareprogramming or software elements, the inventive concept may beimplemented with any programming or scripting language such as C, C++,Java, assembler language, or the like, with the various algorithms beingimplemented with any combination of data structures, objects, processes,routines or other programming elements. Functional aspects may beimplemented in algorithms that are executed on one or more processors.Furthermore, inventive concepts could employ any number of conventionaltechniques for electronics configuration, signal processing and/orcontrol, data processing and the like. The words “mechanism”, “element”,“unit”, and “configuration” are used broadly and are not limited tomechanical or physical embodiments, but can include software routines inconjunction with processors, etc.

The particular implementations shown and described herein areillustrative examples of inventive concepts and are not intended tootherwise limit the scope of inventive concepts in any way. For the sakeof brevity, conventional electronics, control systems, softwaredevelopment and other functional aspects of the systems may not bedescribed in detail. Furthermore, the connecting lines, or connectorsshown in the various figures presented are intended to represent examplefunctional relationships and/or physical or logical couplings betweenthe various elements. It should be noted that many alternative oradditional functional relationships, physical connections or logicalconnections may be present in a practical device. Moreover, no item orcomponent is essential to the practice of inventive concepts unless theelement is specifically described as “essential” or “critical”.

The use of the terms “a”, “an”, and “the” and similar referents in thecontext of describing inventive concepts (especially in the context ofthe following claims) are to be construed to cover both the singular andthe plural. Furthermore, recitation of ranges of values herein aremerely intended to serve as a shorthand method of referring individuallyto each separate value falling within the range, unless otherwiseindicated herein, and each separate value is incorporated into thespecification as if it were individually recited herein. Also, the stepsof all methods described herein can be performed in any suitable orderunless otherwise indicated herein or otherwise clearly contradicted bycontext. Inventive concepts are not limited to the described order ofthe steps. The use of any and all examples, or example language (e.g.,“such as”) provided herein, is intended merely to better illuminateinventive concepts and does not pose a limitation on the scope ofinventive concepts unless otherwise claimed. It will be understood byone of ordinary skill in the art that numerous modifications andadaptations will be made according to design conditions and factorswithout departing from the spirit and scope of inventive concepts.

What is claimed is:
 1. An apparatus for recognizing a movement of asubject, the apparatus comprising: a light source configured to emitlight to the subject; a single image sensor configured to receive lightreflected from the subject; and a processor configured to, detect apixel that receives the reflected light, the pixel being in a pixelarray of the single image sensor, and determine a direction in which thesubject moves based on a change in a position of the detected pixel inthe single image sensor.
 2. The apparatus of claim 1, wherein theprocessor is configured to determine the direction in which the subjectmoves according to a direction of the change in the position of thedetected pixel in the single image sensor.
 3. The apparatus of claim 1,wherein the processor is configured to, during a period of time, detectthe pixel at a first time and fail to detect the pixel at a second time,and determine the direction in which the subject moves during the periodof time.
 4. The apparatus of claim 1, wherein the processor isconfigured to determine a distance by which the subject moves based onthe change in the position of the detected pixel in the single imagesensor.
 5. The apparatus of claim 4, wherein the processor is configuredto determine the distance by which the subject moves based on an amountof the change in the position of the detected pixel in the single imagesensor and an angle at which the light source emits the light.
 6. Theapparatus of claim 1, wherein the processor is configured to recognize apattern according to the change in the position of the detected pixel inthe single image sensor.
 7. The apparatus of claim 1, furthercomprising: a filter configured to filter the reflected light totransmit light having a desired wavelength, wherein the processor isconfigured to detect the pixel using the transmitted light having thedesired wavelength.
 8. The apparatus of claim 1, further comprising: alens configured to collect the reflected light such that the singleimage sensor receives the reflected light collected by the lens.
 9. Awearable device for recognizing a movement of a subject, the wearabledevice comprising: a light source configured to emit light to thesubject; an image sensor configured to receive light reflected from thesubject; and a processor configured to, detect a pixel that receives thereflected light, the pixel being in a pixel array of the image sensor,and track a movement of the subject based on a change in a position ofthe detected pixel, wherein the light source and the image sensor arearranged on a side of the wearable device.
 10. The wearable device ofclaim 9, wherein the image sensor is a single image sensor.
 11. Thewearable device of claim 9, wherein the processor is configured todetermine a direction in which the subject moves based on a change in aposition of the detected pixel in the image sensor.
 12. The wearabledevice of claim 9, wherein the processor is configured to determine adistance by which the subject moves based on the change in the positionof the detected pixel in the image sensor.
 13. A method of recognizing amovement of a subject, the method comprising: emitting, by a lightsource, light to the subject; receiving, by a single image sensor, lightreflected from the subject; detecting a pixel that receives thereflected light, the pixel being in a pixel array of the single imagesensor; and determining a direction in which the subject moves based ona change in a position of the detected pixel in the single image sensor.14. The method of claim 13, wherein determining comprises: determiningthe direction in which the subject moves according to a direction of thechange in the position of the detected pixel in the single image sensor.15. The method of claim 13, wherein the determining comprises: during aperiod of time, detecting the pixel at a first time and failing todetect the pixel at a second time, and determining the direction inwhich the subject moves during the period of time.
 16. The method ofclaim 13, further comprising: determining a distance by which thesubject moves based on the change in the position of the detected pixelin the single image sensor.
 17. The method of claim 16, whereindetermining the distance comprises: determining the distance by whichthe subject moves based on an amount of the change in the position ofthe detected pixel in the single image sensor, and an angle at which thelight source emits the light.
 18. The method of claim 13, furthercomprising: recognizing a pattern according to the change in theposition of the detected pixel in the single image sensor.
 19. Themethod of claim 13, further comprising: filtering the reflected light totransmit light having a desired wavelength, wherein the detectingcomprises detecting the pixel using the transmitted light having thedesired wavelength.
 20. The method of claim 13, wherein the receivinglight reflected from the subject comprises: receiving the lightreflected from the subject through a lens.